Network security products such as intrusion detection systems (ID systems) and firewalls can use a passive filtering technique to detect policy violations and patterns of misuse upon networks to which the Security products are coupled. The passive filtering technique usually comprises monitoring traffic upon the network for packets of data. A signature analysis or pattern matching algorithm is used upon the packets, wherein the packets are compared to “attack signatures”, or signatures of known policy violations or patterns of misuse.
In order to properly detect policy violations and patterns of misuse, security products often must place the packets of data in contexts relevant to such connection criteria as space, time, and event. Space is usually defined in terms of a source-destination connection at the port level. Time is defined as the amount of time to continue associating packets for the type of connection defined by the source-destination connection. Event is defined as a type of connection, which in turn defines the types of policy and misuse signatures that can occur with each packet. As the size of a network expands, there are greater numbers of connections which leads to greater numbers of lookups and comparisons that must be performed by the Security product.
Two problems are associated with conventional security products. First, conventional security products have insufficient information to self-configure for reliable detection of policy violations and patterns of misuse. For example, conventional security products have no mechanism to reliably ascertain network information of the network to which the security product is coupled. This leads to such disadvantages such as being unable to accurately predict the effect of a particular packet upon a destination device. Furthermore, a conventional security product has no mechanism to ascertain the network topology and thus cannot predict if a certain packet will reach its intended destination. Such a lack of network information compromises the security product's ability to detect such attacks such as insertion attacks, evasion attacks and denial of service attacks. Some of these problems with conventional security products are documented by Ptacek and Newsham, Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection, Secure Networks Incorporated, January 1998.
A second problem associated with conventional security products is the result of scarcity of processor and memory resources. Conventional security products may begin to drop packets and shut down certain tasks in an unpredictable fashion once the system depletes its memory or processor resources. As the size of a network grows, such a failure becomes more likely, as the greater the number of connections onto the network requires a greater number of lookups and comparisons performed by the Security product. Additionally, an increase in number and complexity of the types of misuse the security product is required to detect can further degrade performance. An increase in traffic flow further drains a security product's resources. For example, conventional ID systems cannot operate effectively at high network bandwidth utilization.
Some conventional systems have attempted to achieve performance gains by decreasing the number of misuse signatures the security product monitors. Fewer signatures translate into fewer memory comparisons for each packet that flows through the security product. However, such a solution makes a network more vulnerable to attacks.
Other conventional systems rely on the user to enumerate the network information, such as the types of operating systems and applications running on the protected network. These systems then disable certain misuse signatures accordingly.
Such a conventional solution, however, introduces its own problems. For example, if the user provides an inaccurate assessment of the network, then incorrect signatures may be disabled, meaning that undetected policy violations and network attacks are possible. Additionally, networks are rarely stable environments and the addition or deletion of devices or services can make the original network information supplied by the user inaccurate.
A further disadvantage of such conventional security products is that they are not designed to function in an environment wherein the traffic exceeds their memory or processor capacity. Such conventional systems, when confronted with traffic that exceeds their capacity, may start dropping packets and degrade performance in an unpredictable fashion. This can lead to an unknown security posture or profile, which can leave a network more vulnerable to undetected attacks.